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DETAILED ACTION 

Claim Rejections - 35 USC §112 

1. The following is a quotation of the first paragraph of 35 U.S. C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

2. Claim 42 is rejected under 35 U.S.C. 112, first paragraph, as failing to comply 
with the written description requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to reasonably convey to one 
skilled in the relevant art that the inventor(s), at the time the application was filed, had 
possession of the claimed invention. Claim 42 cites "verifying that an agent requested a 
customer's permission to put the customer on hold" without any clear support in the 
specification. 

Response to Arguments 

3. Applicants arguments with respect to claims 1,19, and 34-45 have been 
considered but are moot in view of the new grounds of rejection. Eilbacher et al. US 
6724887 B1 (hereinafter Eilbacher) has been incorporated to address analysis of a 
captured interaction within a time sequence/limit relative to a media type. 

NOTE: In view of the present invention, Examiner would like to point out the definition of 
media as defined in the specification, wherein stated, "The media could be audio, video 
and other multi-media". Examiner therefore has maintained the use of Tsuchikawa; et 
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al. US 5748775 A (hereinafter Tsuchikawa) in view of Soundararajan US 20030106072 
A1 (hereinafter Soundararajan), as both references incorporate the use of media and 
media selection. Further Tsuchikawa teaches the capturing of time/temporal positions 
of media (i.e. frames), wherein this method is functionally equivalent and equally 
effective to a "pivot spot definer" as taught in the present invention. 

Further, Examiner has incorporated Eilbacher to address interaction within a time 
sequence, such as interaction captured between users or between a user and a service, 
agent, or call handling system. Eilbacher teaches the capturing of interactions in a 
media type environment. The combination of Tsuchikawa, Soundararajan, in view of 
Eilbacher now addresses various forms of media and time based selection of captured 
interactions. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1-9,11,13,16,1 9-25, 31 , 33-39, 41 , 42, 44, and 45 are rejected under 
35 U.S.C. 103(a) as being unpatentable over Tsuchikawa et al. US 5748775 A 
(hereinafter Tsuchikawa) in view of Soundararajan US 20030106072 A1 (hereinafter 
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Soundararajan) and further in view of Eilbacher et al. US 6724887 B1 (hereinafter 
Eilbacher). 

Re claim 1 , Tsuchikawa teaches an apparatus for event-driven content analysis 
of a captured interaction, within a computerized system having a processing unit and a 
storage unit (Col. 2 line 55 - Col. 3 line 13), the apparatus comprising the elements of: 

a pivot spot definer component to mark an at least one time position in the 
interaction media to indicate the occurrence of an at least one pre-defined event or data 
item (Col. 5 lines 18-26); 

a region of interest definer component to define a region of interest to determine 
the time limits of an at least one segment of the interaction, the segment containing the 
time position of the pivot spot (Col. 5 lines 27-51) 

However, Tsuchikawa fails to teach a media type selector component to select a 
type of an interaction media inputted for analysis from an at least one interaction 
recording or storage device; 

Soundararajan teaches a multimedia source input for receiving multimedia 
content from a multimedia stream 102. The term "multimedia stream" as used herein is 
intended to refer to signals that may originate from a cable, satellite, or terrestrial 
broadcast, or from an alternative multiple-program stream. The multimedia content 
preferably includes, but is not limited to, broadcast television video signals, such as 
National Television Standards Committee (NTSC) signals, DTV signals, and high 
definition television (HDTV) signals. The multimedia content may also include still 
images, audio signals (e.g., from a satellite radio source), etc. The STB/DTV 104 may 
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include multiple multimedia inputs and a multiplexer or equivalent switching circuitry (not 
shown) for selecting one of a plurality of multimedia sources (e.g., video camera, digital 
still camera, etc.) (Soundararajan [0019]). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawato incorporate teach a media 
type selector component to select a type of an interaction media inputted for analysis 
from an at least one interaction recording or storage device as taught by Soundararajan 
for the purposes of a universal control system fro various multimedia equipment/data, 
wherein all multimedia can interface through a single distributed storage medium 
(Soundararajan [0019]). 

However, Tsuchikawa in view of Soundararajan fails to teach a captured 
interaction 

Eilbacher teaches a contact center 200 of FIG. 2, and in particular a telephone 
call center. Referring to FIG. 3, customers 100 access the contact center through the 
public switched telephone network (PSTN) 101 and an automatic call distribution 
system 102 (PBX/ACD) directs the communication to one of a plurality of agent work 
stations 104. Each agent work station 104 includes, for example, a computer and a 
telephone set. Communications are directed to the agent stations 104 based on the 
availability of the agent. In those contact centers handling communications for a 
number of different clients, communications to a particular client may be routed to a 
finite group of agents specifically trained to respond to the needs of that customer or 
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that client. Alternatively, the PBX/ACD 102 may include an interactive voice response 
(IVR) system that presents an audio menu to a customer, requesting a response by way 
of the customer's telephone key pad or by way of a voice response. Then, a call is 
directed to a particular group of agent stations 104 or to a particular information retrieval 
system, based on the responses of the customer. For example, the system can provide 
the customer 100 with the address to which products should be returned or the Internet 
address for obtaining additional product information. All data associated with the 
customer's communication and the agents responsive interaction with the customer may 
be recorded by a monitor module 210 within monitoring system 204. Examples of the 
data typically recorded by a telephone call center system include the audio 
communication between the customer and the agent, key pad data input by the 
customer, screens viewed by the agent on the computer at the agent station 104 
(carried by data line 105), the start and end time for the customer's communication, the 
identity of the customer, including the originating telephone number and the called 
number, the identity of the various agents servicing the communications, the length of 
time the customer is on hold and the steps the customer navigated before terminating 
the communication (Eilbacher Col. 8 lines 29-67). 

Further, Eilbacher teaches incoming and outgoing calls can be recorded in their 
entirety; particular calls can be identified for recording, such as by client or agent; and 
calls can be recorded by event, such as calls exceeding five minutes. If "cradle-to- 
grave" recording is used, then all information related to a particular telephone call or 
caller-initiated transaction is recorded, from the time the call enters the contact center to 
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the later of: the caller hanging up or the agent completing the transaction. All of the 
interactions during the call are recorded, including interaction with the IVR system, time 
spent on hold, data keyed through the caller's key pad, conversations with the agent, 
and screens displayed by the agent at his/her station 104 during the transaction. These 
types of recordings allow for evaluation of the full customer experience throughout the 
transaction. As an example, the length of time a customer was on hold during a 
purchase transaction can be analyzed as a possible deterrent to completing a purchase. 
Such information may be used by contact center managers to modify their procedures, 
staffing, and/or equipment to improve the customer's experience when using the contact 
center. The comprehensiveness of the data capture of the present invention also allows 
for the subsequent verification of transaction content. For example, a dispute over what 
information was verbally provided by a caller applying for insurance coverage over the 
telephone can easily be resolved by replaying the application call in its entirety. 
Whether a customer selected size 10 can also be proven, as can whether the 
customer/investor authorized the purchase of 100 shares of a particular stock. 
(Eilbacher Col. 9 lines 10-39). 

Furthermore, Eilbacher teaches types of parameters which can be analyzed by 
the customer experience analyzing unit 208 include the number of key strokes entered 
by the customer during a telephone call, the length of a telephone call, time on hold, 
number of transfers, or length of a queue. That is, if the length of the telephone call, the 
number of key strokes entered during the call or the length of a queue exceeded 
predetermined levels, the customer experience analyzing unit 208 can determine that 
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the communication was likely unsatisfactory. In addition, speech detection or word 
spotting can be used to detect certain inflammatory words such as curse words. For 
example, in the case of word spotting, an analysis is performed on recorded audio such 
as a telephone call. The audio is automatically processed, searching for any key words 
on a predefined list which have been identified as cause for concern. If any of the 
words are found, the call is marked as a potentially negative customer experience. This 
word spotting analysis can be done separately, or in addition to the stress analysis. 
Similarly, in connection with an e-mail communication, a text search can be used to look 
for words such as curse words, which might tend to indicate an unsatisfactory customer 
experience (Eilbacher Col. 1 1 lines 25-61 ). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan to 
incorporate event-driven content analysis of a captured interaction as taught by 
Eilbacher to allow for the monitoring of a customer and agent interaction, wherein 
various retrieval methods are implemented based on responses (Eilbacher Col. 8 lines 
29-67) and key locations of media selected based on the customer responses (i.e. 
images, keywords, etc.) (Eilbacher Col. 11 lines 25-61). 

Re claim 2, Tsuchikawa teaches the apparatus of claim 1 further comprising a 
content analysis input selector component to determine an at least one input or 
parameter for an at least one analyzer component (Col. 5 lines 5-1 7). 
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Re claim 3, Tsuchikawa teaches the apparatus of claim 1 further comprises an 
analysis type selector component to identify and to select an at least one analyzer 
component type for determining the Region of Interest (Col. 5 lines 27-51 ). 

Re claim 4, Tsuchikawa teaches the apparatus of claim 1 further comprising an 
audio analyzer component for performing an analysis on the media selected by the 
media selector component in a location adjacent to the pivot spot identified by the pivot 
spot defined component (Col. 5 lines 5-17). 

However, Tsuchikawa fails to teach a media selector component (Soundararajan 
[0019]); 

Soundararajan teaches a multimedia source input for receiving multimedia 
content from a multimedia stream 102. The term "multimedia stream" as used herein is 
intended to refer to signals that may originate from a cable, satellite, or terrestrial 
broadcast, or from an alternative multiple-program stream. The multimedia content 
preferably includes, but is not limited to, broadcast television video signals, such as 
National Television Standards Committee (NTSC) signals, DTV signals, and high 
definition television (HDTV) signals. The multimedia content may also include still 
images, audio signals (e.g., from a satellite radio source), etc. The STB/DTV 104 may 
include multiple multimedia inputs and a multiplexer or equivalent switching circuitry (not 
shown) for selecting one of a plurality of multimedia sources (e.g., video camera, digital 
still camera, etc.). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawato incorporate a media type 
selector to select a type of interaction media inputted for analysis on a frame by frame 
basis as taught by Soundararajan for the purposes of a universal control system fro 
various multimedia equipment/data, wherein all multimedia can interface through a 
single distributed storage medium (Soundararajan [0019]). 

Re claim 5, Tsuchikawa teaches the apparatus of claim 1 further comprising an 
analyzer component for performing an analysis on the media selected by the media 
selector component in a location adjacent to the pivot spot identified by the pivot spot 
definer component (Col. 5 lines 5-17) using an analyzer selected by the analysis type 
selector using parameters fed to, or selected by the content analysis input selector 
component (Col. 5 lines 27-51). 

However, Tsuchikawa fails to teach a media selector component (Soundararajan 
[0019]); 

Soundararajan teaches a multimedia source input for receiving multimedia 
content from a multimedia stream 102. The term "multimedia stream" as used herein is 
intended to refer to signals that may originate from a cable, satellite, or terrestrial 
broadcast, or from an alternative multiple-program stream. The multimedia content 
preferably includes, but is not limited to, broadcast television video signals, such as 
National Television Standards Committee (NTSC) signals, DTV signals, and high 
definition television (HDTV) signals. The multimedia content may also include still 
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images, audio signals (e.g., from a satellite radio source), etc. The STB/DTV 104 may 
include multiple multimedia inputs and a multiplexer or equivalent switching circuitry (not 
shown) for selecting one of a plurality of multimedia sources (e.g., video camera, digital 
still camera, etc.). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawato incorporate a media type 
selector to select a type of interaction media inputted for analysis on a frame by frame 
basis as taught by Soundararajan for the purposes of a universal control system fro 
various multimedia equipment/data, wherein all multimedia can interface through a 
single distributed storage medium (Soundararajan [0019]). 

Re claim 6, Tsuchikawa teaches the apparatus of claim 1 further comprising an 
analyzer component for performing an analysis on the media selected by the media 
selector component within the region of interest identified by the region of interest 
definer component (Col. 5 lines 27-51 ) using an analyzer selected by the analysis type 
selector using parameters fed to (Col. 5 lines 5-17), or selected by the content analysis 
input selector component 

However, Tsuchikawa fails to teach a media selector component (Soundararajan 
[0019]); 

Soundararajan teaches a multimedia source input for receiving multimedia 
content from a multimedia stream 102. The term "multimedia stream" as used herein is 
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intended to refer to signals that may originate from a cable, satellite, or terrestrial 
broadcast, or from an alternative multiple-program stream. The multimedia content 
preferably includes, but is not limited to, broadcast television video signals, such as 
National Television Standards Committee (NTSC) signals, DTV signals, and high 
definition television (HDTV) signals. The multimedia content may also include still 
images, audio signals (e.g., from a satellite radio source), etc. The STB/DTV 104 may 
include multiple multimedia inputs and a multiplexer or equivalent switching circuitry (not 
shown) for selecting one of a plurality of multimedia sources (e.g., video camera, digital 
still camera, etc.). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawato incorporate a media type 
selector to select a type of interaction media inputted for analysis on a frame by frame 
basis as taught by Soundararajan for the purposes of a universal control system fro 
various multimedia equipment/data, wherein all multimedia can interface through a 
single distributed storage medium (Soundararajan [0019]). 

Re claims 7 and 31 , Tsuchikawa teaches the apparatus of claim 1 wherein the 
region of interest defined by the region of interest definer component further comprises 
an optimization component for optimizing the region of interest (Col. 5 lines 27-51 & Fig. 
3 item 200) 
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Re claim 8, Tsuchikawa teaches an apparatus of claim 1 further comprises a 
content analysis inputs table to hold in storage the at least one selectable input values 
(Col. 5 lines 5-17). 

Re claim 9, Tsuchikawa teaches the apparatus of claim 1 further comprises the 
element of an audio analyzer component to analyze the audio elements of the 
interaction data (Col. 5 lines 5-17). 

Re claim 1 1 , Tsuchikawa teaches the apparatus of claim 1 further comprises a 
screen event analyzer component to identify and capture an at least one screen and an 
at least one screen event associated with the interaction data and capture at least one 
screen based on the screen event (Col. 5 lines 5-17 & Fig. 4). 

Re claim 13, Tsuchikawa teaches the apparatus of claim 1 further comprising an 
analysis module for performing an analysis on the media (Col. 5 lines 5-17 & Fig. 4). 

Re claim 16, Tsuchikawa teaches the apparatus of claim 1 wherein the region of 
interest is a specific segment of the interaction media that is analyzed to extract 
meaningful interaction-specific information in an organization (Col. 5 lines 27-51). 

Re claim 20, Tsuchikawa fails to teach selecting an interaction media to analyze 
(Soundararajan [0019]); 
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Soundararajan teaches a multimedia source input for receiving multimedia 
content from a multimedia stream 102. The term "multimedia stream" as used herein is 
intended to refer to signals that may originate from a cable, satellite, or terrestrial 
broadcast, or from an alternative multiple-program stream. The multimedia content 
preferably includes, but is not limited to, broadcast television video signals, such as 
National Television Standards Committee (NTSC) signals, DTV signals, and high 
definition television (HDTV) signals. The multimedia content may also include still 
images, audio signals (e.g., from a satellite radio source), etc. The STB/DTV 104 may 
include multiple multimedia inputs and a multiplexer or equivalent switching circuitry (not 
shown) for selecting one of a plurality of multimedia sources (e.g., video camera, digital 
still camera, etc.). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawato incorporate selecting an 
interaction media to analyze as taught by Soundararajan for the purposes of a universal 
control system fro various multimedia equipment/data, wherein all multimedia can 
interface through a single distributed storage medium (Soundararajan [0019]). 

Re claim 21 , Tsuchikawa teaches the method of claim 19 further comprising the 
step of selecting a method for the analysis of the at least one interaction media based 
on the at least one event associated with the interaction (Fig. 3). 
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Re claim 22, Tsuchikawa teaches the method of claim 19 further comprising the 
step of selecting a method for the analysis of the at least one interaction media based 
on the result of a previously performed analysis (Fig. 6 item 130) 

Re claim 23, Tsuchikawa teaches the method of claim 19 further comprising the 
step of selecting the parameters to be used in the at least one analysis instruction step 
on the at least one segment of the interaction media (Col. 5 lines 5-17). 

Re claim 24, Tsuchikawa teaches the method of claim 19 further comprising the 
step of optimizing the region of interest by performing an at least one analysis 
instruction step within the region of interest (Col. 5 lines 27-51) and readjusting the 
region of interest in accordance with the results of the at least one analysis instruction 
step (Fig. 3). 

Re claim 25, Tsuchikawa teaches the method of claim 19 wherein the region of 
interest is predetermined by a user or an apparatus (Col. 5 lines 27-51). 

Re claim 33, Tsuchikawa teaches the method of claim 19 further comprising the 
step of adjusting the at least one pivot spot (Fig. 4) or region of interest on the 
interaction media (Col. 5 lines 27-51). 
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Re claims 19, 44, and 45, Tsuchikawa teaches a method for event-driven content 
analysis, within a computerized system having a processing unit and a storage unit, the 
method comprising the steps of: 

determining an at least one pivot spot, being a time position, on an interaction 
media associated with an at least one event associated with the interaction (Col. 5 lines 
18-26); 

determining the time limits of the at least one segment of the interaction media to 
be analyzed, said limits defining an initial region of interest within the interaction (Col. 5 
lines 27-51); 

optimizing the initial region of interest by performing an at least one analysis 
instruction step within the initial region of interest and readjusting the initial region of 
interest in accordance with a result of the at least one analysis instruction step, to obtain 
a region of interest (Col. 5 lines 27-51 , and Fig. 3 item 200); 

However, Tsuchikawa fails to teach selecting a first method for the analysis of 
the at least one interaction media based on the at least one event associated with the 
interaction; 

selecting a second method for analyzing the region of interest; an 
analyzing the region of interest using the second method 
Soundararajan teaches a multimedia source input for receiving multimedia 
content from a multimedia stream 102. The term "multimedia stream" as used herein is 
intended to refer to signals that may originate from a cable, satellite, or terrestrial 



Application/Control Number: 10/706,282 Page 17 

Art Unit: 2626 

broadcast, or from an alternative multiple-program stream. The multimedia content 
preferably includes, but is not limited to, broadcast television video signals, such as 
National Television Standards Committee (NTSC) signals, DTV signals, and high 
definition television (HDTV) signals. The multimedia content may also include still 
images, audio signals (e.g., from a satellite radio source), etc. The STB/DTV 104 may 
include multiple multimedia inputs and a multiplexer or equivalent switching circuitry (not 
shown) for selecting one of a plurality of multimedia sources (e.g., video camera, digital 
still camera, etc.) (Soundararajan [0019]). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawato incorporate selecting a first 
method for the analysis of the at least one interaction media based on the at least one 
event associated with the interaction, selecting a second method for analyzing the 
region of interest, and analyzing the region of interest using the second method a media 
type selector component to select a type of an interaction media inputted for analysis 
from an at least one interaction recording or storage device as taught by Soundararajan 
for the purposes of a universal control system fro various multimedia equipment/data, 
wherein all multimedia can interface through a single distributed storage medium 
(Soundararajan [0019]). 

However, Tsuchikawa in view of Soundararajan fails to teach interaction media 
receiving interaction data and associated meta-data from an at least one 
interaction; 
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readjusting the initial region of interest in accordance with a result of the at least 
one analysis instruction step, to obtain a region of interest. 

Eilbacher teaches a contact center 200 of FIG. 2, and in particular a telephone 
call center. Referring to FIG. 3, customers 100 access the contact center through the 
public switched telephone network (PSTN) 101 and an automatic call distribution 
system 1 02 (PBX/ACD) directs the communication to one of a plurality of agent work 
stations 104. Each agent work station 104 includes, for example, a computer and a 
telephone set. Communications are directed to the agent stations 104 based on the 
availability of the agent. In those contact centers handling communications for a 
number of different clients, communications to a particular client may be routed to a 
finite group of agents specifically trained to respond to the needs of that customer or 
that client. Alternatively, the PBX/ACD 102 may include an interactive voice response 
(IVR) system that presents an audio menu to a customer, requesting a response by way 
of the customer's telephone key pad or by way of a voice response. Then, a call is 
directed to a particular group of agent stations 104 or to a particular information retrieval 
system, based on the responses of the customer. For example, the system can provide 
the customer 100 with the address to which products should be returned or the Internet 
address for obtaining additional product information. All data associated with the 
customer's communication and the agents responsive interaction with the customer may 
be recorded by a monitor module 210 within monitoring system 204. Examples of the 
data typically recorded by a telephone call center system include the audio 
communication between the customer and the agent, key pad data input by the 
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customer, screens viewed by the agent on the computer at the agent station 104 
(carried by data line 105), the start and end time for the customer's communication, the 
identity of the customer, including the originating telephone number and the called 
number, the identity of the various agents servicing the communications, the length of 
time the customer is on hold and the steps the customer navigated before terminating 
the communication (Eilbacher Col. 8 lines 29-67). 

Further, Eilbacher teaches incoming and outgoing calls can be recorded in their 
entirety; particular calls can be identified for recording, such as by client or agent; and 
calls can be recorded by event, such as calls exceeding five minutes. If "cradle-to- 
grave" recording is used, then all information related to a particular telephone call or 
caller-initiated transaction is recorded, from the time the call enters the contact center to 
the later of: the caller hanging up or the agent completing the transaction. All of the 
interactions during the call are recorded, including interaction with the IVR system, time 
spent on hold, data keyed through the caller's key pad, conversations with the agent, 
and screens displayed by the agent at his/her station 104 during the transaction. These 
types of recordings allow for evaluation of the full customer experience throughout the 
transaction. As an example, the length of time a customer was on hold during a 
purchase transaction can be analyzed as a possible deterrent to completing a purchase. 
Such information may be used by contact center managers to modify their procedures, 
staffing, and/or equipment to improve the customer's experience when using the contact 
center. The comprehensiveness of the data capture of the present invention also allows 
for the subsequent verification of transaction content. For example, a dispute over what 
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information was verbally provided by a caller applying for insurance coverage over the 
telephone can easily be resolved by replaying the application call in its entirety. 
Whether a customer selected size 10 can also be proven, as can whether the 
customer/investor authorized the purchase of 100 shares of a particular stock. 
(Eilbacher Col. 9 lines 10-39). 

Furthermore, Eilbacher teaches types of parameters which can be analyzed by 
the customer experience analyzing unit 208 include the number of key strokes entered 
by the customer during a telephone call, the length of a telephone call, time on hold, 
number of transfers, or length of a queue. That is, if the length of the telephone call, the 
number of key strokes entered during the call or the length of a queue exceeded 
predetermined levels, the customer experience analyzing unit 208 can determine that 
the communication was likely unsatisfactory. In addition, speech detection or word 
spotting can be used to detect certain inflammatory words such as curse words. For 
example, in the case of word spotting, an analysis is performed on recorded audio such 
as a telephone call. The audio is automatically processed, searching for any key words 
on a predefined list which have been identified as cause for concern. If any of the 
words are found, the call is marked as a potentially negative customer experience. This 
word spotting analysis can be done separately, or in addition to the stress analysis. 
Similarly, in connection with an e-mail communication, a text search can be used to look 
for words such as curse words, which might tend to indicate an unsatisfactory customer 
experience (Eilbacher Col. 11 lines 25-61 ). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan to 
incorporate interaction media receiving interaction data and associated meta-data from 
an at least one interaction and readjusting the initial region of interest in accordance 
with a result of the at least one analysis instruction step, to obtain a region of interest as 
taught by Eilbacher to allow for the monitoring of a customer and agent interaction, 
wherein various retrieval methods are implemented based on responses (Eilbacher Col. 
8 lines 29-67) and key locations of media selected based on the customer responses 
(i.e. images, keywords, etc.) to create and ideal/optimal retrieval of information 
(Eilbacher Col. 11 lines 25-61). 

Re claims 34 and 35, Tsuchikawa in view of Soundararajan fails to teach the 
apparatus of claim 1 wherein the pivot spot is determined using at least one item 
selected from the group consisting of: a Computer Telephony Integration event; a 
screen event; an emotional level; and a spotted word. 

Eilbacher teaches a contact center 200 of FIG. 2, and in particular a telephone 
call center. Referring to FIG. 3, customers 100 access the contact center through the 
public switched telephone network (PSTN) 101 and an automatic call distribution 
system 1 02 (PBX/ACD) directs the communication to one of a plurality of agent work 
stations 104. Each agent work station 104 includes, for example, a computer and a 
telephone set. Communications are directed to the agent stations 104 based on the 
availability of the agent. In those contact centers handling communications for a 
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number of different clients, communications to a particular client may be routed to a 
finite group of agents specifically trained to respond to the needs of that customer or 
that client. Alternatively, the PBX/ACD 102 may include an interactive voice response 
(IVR) system that presents an audio menu to a customer, requesting a response by way 
of the customer's telephone key pad or by way of a voice response. Then, a call is 
directed to a particular group of agent stations 104 or to a particular information retrieval 
system, based on the responses of the customer. For example, the system can provide 
the customer 100 with the address to which products should be returned or the Internet 
address for obtaining additional product information. All data associated with the 
customer's communication and the agents responsive interaction with the customer may 
be recorded by a monitor module 210 within monitoring system 204. Examples of the 
data typically recorded by a telephone call center system include the audio 
communication between the customer and the agent, key pad data input by the 
customer, screens viewed by the agent on the computer at the agent station 104 
(carried by data line 105), the start and end time for the customer's communication, the 
identity of the customer, including the originating telephone number and the called 
number, the identity of the various agents servicing the communications, the length of 
time the customer is on hold and the steps the customer navigated before terminating 
the communication (Eilbacher Col. 8 lines 29-67). 

Further, Eilbacher teaches incoming and outgoing calls can be recorded in their 
entirety; particular calls can be identified for recording, such as by client or agent; and 
calls can be recorded by event, such as calls exceeding five minutes. If "cradle-to- 
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grave" recording is used, then all information related to a particular telephone call or 
caller-initiated transaction is recorded, from the time the call enters the contact center to 
the later of: the caller hanging up or the agent completing the transaction. All of the 
interactions during the call are recorded, including interaction with the IVR system, time 
spent on hold, data keyed through the caller's key pad, conversations with the agent, 
and screens displayed by the agent at his/her station 104 during the transaction. These 
types of recordings allow for evaluation of the full customer experience throughout the 
transaction. As an example, the length of time a customer was on hold during a 
purchase transaction can be analyzed as a possible deterrent to completing a purchase. 
Such information may be used by contact center managers to modify their procedures, 
staffing, and/or equipment to improve the customer's experience when using the contact 
center. The comprehensiveness of the data capture of the present invention also allows 
for the subsequent verification of transaction content. For example, a dispute over what 
information was verbally provided by a caller applying for insurance coverage over the 
telephone can easily be resolved by replaying the application call in its entirety. 
Whether a customer selected size 10 can also be proven, as can whether the 
customer/investor authorized the purchase of 100 shares of a particular stock. 
(Eilbacher Col. 9 lines 10-39). 

Furthermore, Eilbacher teaches types of parameters which can be analyzed by 
the customer experience analyzing unit 208 include the number of key strokes entered 
by the customer during a telephone call, the length of a telephone call, time on hold, 
number of transfers, or length of a queue. That is, if the length of the telephone call, the 
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number of key strokes entered during the call or the length of a queue exceeded 
predetermined levels, the customer experience analyzing unit 208 can determine that 
the communication was likely unsatisfactory. In addition, speech detection or word 
spotting can be used to detect certain inflammatory words such as curse words. For 
example, in the case of word spotting, an analysis is performed on recorded audio such 
as a telephone call. The audio is automatically processed, searching for any key words 
on a predefined list which have been identified as cause for concern. If any of the 
words are found, the call is marked as a potentially negative customer experience. This 
word spotting analysis can be done separately, or in addition to the stress analysis. 
Similarly, in connection with an e-mail communication, a text search can be used to look 
for words such as curse words, which might tend to indicate an unsatisfactory customer 
experience (Eilbacher Col. 11 lines 25-61 ). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan to 
incorporate the pivot spot is determined using at least one item selected from the group 
consisting of: a Computer Telephony Integration event; a screen event; an emotional 
level; and a spotted word as taught by Eilbacher to allow for the monitoring of a 
customer and agent interaction, wherein various retrieval methods are implemented 
based on responses (Eilbacher Col. 8 lines 29-67) and key locations of media selected 
based on the customer responses (i.e. images, keywords, etc.) to create and 
ideal/optimal retrieval of information (Eilbacher Col. 11 lines 25-61). 
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Re claims 36 and 37, Tsuchikawa in view of Soundararajan fails to teach the 
apparatus of claim 1 wherein optimizing the region of interest is set according to a 
predetermined length, speaker separation, audio analysis, event analysis, a Computer 
Telephony Integration event analysis, CRM event analysis, a screen event; an 
emotional level, or a spotted word. 

Eilbacher teaches a contact center 200 of FIG. 2, and in particular a telephone 
call center. Referring to FIG. 3, customers 100 access the contact center through the 
public switched telephone network (PSTN) 101 and an automatic call distribution 
system 102 (PBX/ACD) directs the communication to one of a plurality of agent work 
stations 104. Each agent work station 104 includes, for example, a computer and a 
telephone set. Communications are directed to the agent stations 104 based on the 
availability of the agent. In those contact centers handling communications for a 
number of different clients, communications to a particular client may be routed to a 
finite group of agents specifically trained to respond to the needs of that customer or 
that client. Alternatively, the PBX/ACD 102 may include an interactive voice response 
(IVR) system that presents an audio menu to a customer, requesting a response by way 
of the customer's telephone key pad or by way of a voice response. Then, a call is 
directed to a particular group of agent stations 104 or to a particular information retrieval 
system, based on the responses of the customer. For example, the system can provide 
the customer 100 with the address to which products should be returned or the Internet 
address for obtaining additional product information. All data associated with the 
customer's communication and the agents responsive interaction with the customer may 
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be recorded by a monitor module 210 within monitoring system 204. Examples of the 
data typically recorded by a telephone call center system include the audio 
communication between the customer and the agent, key pad data input by the 
customer, screens viewed by the agent on the computer at the agent station 104 
(carried by data line 105), the start and end time for the customer's communication, the 
identity of the customer, including the originating telephone number and the called 
number, the identity of the various agents servicing the communications, the length of 
time the customer is on hold and the steps the customer navigated before terminating 
the communication (Eilbacher Col. 8 lines 29-67). 

Further, Eilbacher teaches incoming and outgoing calls can be recorded in their 
entirety; particular calls can be identified for recording, such as by client or agent; and 
calls can be recorded by event, such as calls exceeding five minutes. If "cradle-to- 
grave" recording is used, then all information related to a particular telephone call or 
caller-initiated transaction is recorded, from the time the call enters the contact center to 
the later of: the caller hanging up or the agent completing the transaction. All of the 
interactions during the call are recorded, including interaction with the IVR system, time 
spent on hold, data keyed through the caller's key pad, conversations with the agent, 
and screens displayed by the agent at his/her station 104 during the transaction. These 
types of recordings allow for evaluation of the full customer experience throughout the 
transaction. As an example, the length of time a customer was on hold during a 
purchase transaction can be analyzed as a possible deterrent to completing a purchase. 
Such information may be used by contact center managers to modify their procedures, 
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staffing, and/or equipment to improve the customer's experience when using the contact 
center. The comprehensiveness of the data capture of the present invention also allows 
for the subsequent verification of transaction content. For example, a dispute over what 
information was verbally provided by a caller applying for insurance coverage over the 
telephone can easily be resolved by replaying the application call in its entirety. 
Whether a customer selected size 10 can also be proven, as can whether the 
customer/investor authorized the purchase of 100 shares of a particular stock. 
(Eilbacher Col. 9 lines 10-39). 

Furthermore, Eilbacher teaches types of parameters which can be analyzed by 
the customer experience analyzing unit 208 include the number of key strokes entered 
by the customer during a telephone call, the length of a telephone call, time on hold, 
number of transfers, or length of a queue. That is, if the length of the telephone call, the 
number of key strokes entered during the call or the length of a queue exceeded 
predetermined levels, the customer experience analyzing unit 208 can determine that 
the communication was likely unsatisfactory. In addition, speech detection or word 
spotting can be used to detect certain inflammatory words such as curse words. For 
example, in the case of word spotting, an analysis is performed on recorded audio such 
as a telephone call. The audio is automatically processed, searching for any key words 
on a predefined list which have been identified as cause for concern. If any of the 
words are found, the call is marked as a potentially negative customer experience. This 
word spotting analysis can be done separately, or in addition to the stress analysis. 
Similarly, in connection with an e-mail communication, a text search can be used to look 
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for words such as curse words, which might tend to indicate an unsatisfactory customer 
experience (Eilbacher Col. 11 lines 25-61). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan to 
incorporate optimizing the region of interest is set according to a predetermined length, 
speaker separation, audio analysis, event analysis, a Computer Telephony Integration 
event analysis, CRM event analysis, a screen event; an emotional level, or a spotted 
word as taught by Eilbacher to allow for the monitoring of a customer and agent 
interaction, wherein various retrieval methods are implemented based on responses 
(Eilbacher Col. 8 lines 29-67) and key locations of media selected based on the 
customer responses (i.e. images, keywords, etc.) to create and ideal/optimal retrieval of 
information (Eilbacher Col. 11 lines 25-61). 

Re claims 38 and 39, Tsuchikawa in view of Soundararajan fails to teach the 
apparatus of claim 1 wherein the captured interaction is between an agent and a 
customer. 

Eilbacher teaches a contact center 200 of FIG. 2, and in particular a telephone 
call center. Referring to FIG. 3, customers 100 access the contact center through the 
public switched telephone network (PSTN) 101 and an automatic call distribution 
system 1 02 (PBX/ACD) directs the communication to one of a plurality of agent work 
stations 104. Each agent work station 104 includes, for example, a computer and a 
telephone set. Communications are directed to the agent stations 104 based on the 
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availability of the agent. In those contact centers handling communications for a 
number of different clients, communications to a particular client may be routed to a 
finite group of agents specifically trained to respond to the needs of that customer or 
that client. Alternatively, the PBX/ACD 102 may include an interactive voice response 
(IVR) system that presents an audio menu to a customer, requesting a response by way 
of the customer's telephone key pad or by way of a voice response. Then, a call is 
directed to a particular group of agent stations 104 or to a particular information retrieval 
system, based on the responses of the customer. For example, the system can provide 
the customer 100 with the address to which products should be returned or the Internet 
address for obtaining additional product information. All data associated with the 
customer's communication and the agents responsive interaction with the customer may 
be recorded by a monitor module 210 within monitoring system 204. Examples of the 
data typically recorded by a telephone call center system include the audio 
communication between the customer and the agent, key pad data input by the 
customer, screens viewed by the agent on the computer at the agent station 104 
(carried by data line 105), the start and end time for the customer's communication, the 
identity of the customer, including the originating telephone number and the called 
number, the identity of the various agents servicing the communications, the length of 
time the customer is on hold and the steps the customer navigated before terminating 
the communication (Eilbacher Col. 8 lines 29-67). 

Further, Eilbacher teaches incoming and outgoing calls can be recorded in their 
entirety; particular calls can be identified for recording, such as by client or agent; and 
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calls can be recorded by event, such as calls exceeding five minutes. If "cradle-to- 
grave" recording is used, then all information related to a particular telephone call or 
caller-initiated transaction is recorded, from the time the call enters the contact center to 
the later of: the caller hanging up or the agent completing the transaction. All of the 
interactions during the call are recorded, including interaction with the IVR system, time 
spent on hold, data keyed through the caller's key pad, conversations with the agent, 
and screens displayed by the agent at his/her station 104 during the transaction. These 
types of recordings allow for evaluation of the full customer experience throughout the 
transaction. As an example, the length of time a customer was on hold during a 
purchase transaction can be analyzed as a possible deterrent to completing a purchase. 
Such information may be used by contact center managers to modify their procedures, 
staffing, and/or equipment to improve the customer's experience when using the contact 
center. The comprehensiveness of the data capture of the present invention also allows 
for the subsequent verification of transaction content. For example, a dispute over what 
information was verbally provided by a caller applying for insurance coverage over the 
telephone can easily be resolved by replaying the application call in its entirety. 
Whether a customer selected size 10 can also be proven, as can whether the 
customer/investor authorized the purchase of 100 shares of a particular stock. 
(Eilbacher Col. 9 lines 10-39). 

Furthermore, Eilbacher teaches types of parameters which can be analyzed by 
the customer experience analyzing unit 208 include the number of key strokes entered 
by the customer during a telephone call, the length of a telephone call, time on hold, 
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number of transfers, or length of a queue. That is, if the length of the telephone call, the 
number of key strokes entered during the call or the length of a queue exceeded 
predetermined levels, the customer experience analyzing unit 208 can determine that 
the communication was likely unsatisfactory. In addition, speech detection or word 
spotting can be used to detect certain inflammatory words such as curse words. For 
example, in the case of word spotting, an analysis is performed on recorded audio such 
as a telephone call. The audio is automatically processed, searching for any key words 
on a predefined list which have been identified as cause for concern. If any of the 
words are found, the call is marked as a potentially negative customer experience. This 
word spotting analysis can be done separately, or in addition to the stress analysis. 
Similarly, in connection with an e-mail communication, a text search can be used to look 
for words such as curse words, which might tend to indicate an unsatisfactory customer 
experience (Eilbacher Col. 11 lines 25-61 ). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan to 
incorporate the captured interaction is between an agent and a customer as taught by 
Eilbacher to allow for the monitoring of a customer and agent interaction, wherein 
various retrieval methods are implemented based on responses (Eilbacher Col. 8 lines 
29-67) and key locations of media selected based on the customer responses (i.e. 
images, keywords, etc.) to create and ideal/optimal retrieval of information (Eilbacher 
Col. 11 lines 25-61). 
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Re claim 41 , Tsuchikawa in view of Soundararajan fails to teach the method of 
claim 19 wherein the method is used for detecting customer chum indications, wherein 
the pivot spot is defined using a CTI hold event or a cancellation-related screen event; 
and wherein the region of interest is defined using emotion analysis or word spotting. 

Eilbacher teaches a contact center 200 of FIG. 2, and in particular a telephone 
call center. Referring to FIG. 3, customers 100 access the contact center through the 
public switched telephone network (PSTN) 101 and an automatic call distribution 
system 102 (PBX/ACD) directs the communication to one of a plurality of agent work 
stations 104. Each agent work station 104 includes, for example, a computer and a 
telephone set. Communications are directed to the agent stations 104 based on the 
availability of the agent. In those contact centers handling communications for a 
number of different clients, communications to a particular client may be routed to a 
finite group of agents specifically trained to respond to the needs of that customer or 
that client. Alternatively, the PBX/ACD 102 may include an interactive voice response 
(IVR) system that presents an audio menu to a customer, requesting a response by way 
of the customer's telephone key pad or by way of a voice response. Then, a call is 
directed to a particular group of agent stations 104 or to a particular information retrieval 
system, based on the responses of the customer. For example, the system can provide 
the customer 100 with the address to which products should be returned or the Internet 
address for obtaining additional product information. All data associated with the 
customer's communication and the agents responsive interaction with the customer may 
be recorded by a monitor module 210 within monitoring system 204. Examples of the 
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data typically recorded by a telephone call center system include the audio 
communication between the customer and the agent, key pad data input by the 
customer, screens viewed by the agent on the computer at the agent station 104 
(carried by data line 105), the start and end time for the customer's communication, the 
identity of the customer, including the originating telephone number and the called 
number, the identity of the various agents servicing the communications, the length of 
time the customer is on hold and the steps the customer navigated before terminating 
the communication (Eilbacher Col. 8 lines 29-67). 

Further, Eilbacher teaches incoming and outgoing calls can be recorded in their 
entirety; particular calls can be identified for recording, such as by client or agent; and 
calls can be recorded by event, such as calls exceeding five minutes. If "cradle-to- 
grave" recording is used, then all information related to a particular telephone call or 
caller-initiated transaction is recorded, from the time the call enters the contact center to 
the later of: the caller hanging up or the agent completing the transaction. All of the 
interactions during the call are recorded, including interaction with the IVR system, time 
spent on hold, data keyed through the caller's key pad, conversations with the agent, 
and screens displayed by the agent at his/her station 104 during the transaction. These 
types of recordings allow for evaluation of the full customer experience throughout the 
transaction. As an example, the length of time a customer was on hold during a 
purchase transaction can be analyzed as a possible deterrent to completing a purchase. 
Such information may be used by contact center managers to modify their procedures, 
staffing, and/or equipment to improve the customer's experience when using the contact 
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center. The comprehensiveness of the data capture of the present invention also allows 
for the subsequent verification of transaction content. For example, a dispute over what 
information was verbally provided by a caller applying for insurance coverage over the 
telephone can easily be resolved by replaying the application call in its entirety. 
Whether a customer selected size 10 can also be proven, as can whether the 
customer/investor authorized the purchase of 100 shares of a particular stock. 
(Eilbacher Col. 9 lines 10-39). 

Furthermore, Eilbacher teaches types of parameters which can be analyzed by 
the customer experience analyzing unit 208 include the number of key strokes entered 
by the customer during a telephone call, the length of a telephone call, time on hold, 
number of transfers, or length of a queue. That is, if the length of the telephone call, the 
number of key strokes entered during the call or the length of a queue exceeded 
predetermined levels, the customer experience analyzing unit 208 can determine that 
the communication was likely unsatisfactory. In addition, speech detection or word 
spotting can be used to detect certain inflammatory words such as curse words. For 
example, in the case of word spotting, an analysis is performed on recorded audio such 
as a telephone call. The audio is automatically processed, searching for any key words 
on a predefined list which have been identified as cause for concern. If any of the 
words are found, the call is marked as a potentially negative customer experience. This 
word spotting analysis can be done separately, or in addition to the stress analysis. 
Similarly, in connection with an e-mail communication, a text search can be used to look 
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for words such as curse words, which might tend to indicate an unsatisfactory customer 
experience (Eilbacher Col. 11 lines 25-61). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan to 
incorporate detecting customer chum indications, wherein the pivot spot is defined using 
a CTI hold event or a cancellation-related screen event; and wherein the region of 
interest is defined using emotion analysis or word spotting as taught by Eilbacher to 
allow for the monitoring of a customer and agent interaction, wherein various retrieval 
methods are implemented based on responses (Eilbacher Col. 8 lines 29-67) and key 
locations of media selected based on the customer responses (i.e. images, keywords, 
etc.) to create and ideal/optimal retrieval of information (Eilbacher Col. 1 1 lines 25-61). 

Re claim 42, Tsuchikawa in view of Soundararajan fails to teach the method of 
claim 19 wherein the method is used for verifying that an agent requested a customer's 
permission to put the customer on hold, wherein the pivot spot is the time the agent put 
the customer on hold, the initial region of interest is the whole interaction, and wherein 
the region of interest is defined by a first predetermined number of seconds prior to the 
pivot spot and a second predetermined number of seconds following the hold. 

Eilbacher teaches a contact center 200 of FIG. 2, and in particular a telephone 
call center. Referring to FIG. 3, customers 100 access the contact center through the 
public switched telephone network (PSTN) 101 and an automatic call distribution 
system 1 02 (PBX/ACD) directs the communication to one of a plurality of agent work 
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stations 104. Each agent work station 104 includes, for example, a computer and a 
telephone set. Communications are directed to the agent stations 104 based on the 
availability of the agent. In those contact centers handling communications for a 
number of different clients, communications to a particular client may be routed to a 
finite group of agents specifically trained to respond to the needs of that customer or 
that client. Alternatively, the PBX/ACD 102 may include an interactive voice response 
(IVR) system that presents an audio menu to a customer, requesting a response by way 
of the customer's telephone key pad or by way of a voice response. Then, a call is 
directed to a particular group of agent stations 104 or to a particular information retrieval 
system, based on the responses of the customer. For example, the system can provide 
the customer 100 with the address to which products should be returned or the Internet 
address for obtaining additional product information. All data associated with the 
customer's communication and the agents responsive interaction with the customer may 
be recorded by a monitor module 210 within monitoring system 204. Examples of the 
data typically recorded by a telephone call center system include the audio 
communication between the customer and the agent, key pad data input by the 
customer, screens viewed by the agent on the computer at the agent station 104 
(carried by data line 105), the start and end time for the customer's communication, the 
identity of the customer, including the originating telephone number and the called 
number, the identity of the various agents servicing the communications, the length of 
time the customer is on hold and the steps the customer navigated before terminating 
the communication (Eilbacher Col. 8 lines 29-67). 
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Further, Eilbacher teaches incoming and outgoing calls can be recorded in their 
entirety; particular calls can be identified for recording, such as by client or agent; and 
calls can be recorded by event, such as calls exceeding five minutes. If "cradle-to- 
grave" recording is used, then all information related to a particular telephone call or 
caller-initiated transaction is recorded, from the time the call enters the contact center to 
the later of: the caller hanging up or the agent completing the transaction. All of the 
interactions during the call are recorded, including interaction with the IVR system, time 
spent on hold, data keyed through the caller's key pad, conversations with the agent, 
and screens displayed by the agent at his/her station 104 during the transaction. These 
types of recordings allow for evaluation of the full customer experience throughout the 
transaction. As an example, the length of time a customer was on hold during a 
purchase transaction can be analyzed as a possible deterrent to completing a purchase. 
Such information may be used by contact center managers to modify their procedures, 
staffing, and/or equipment to improve the customer's experience when using the contact 
center. The comprehensiveness of the data capture of the present invention also allows 
for the subsequent verification of transaction content. For example, a dispute over what 
information was verbally provided by a caller applying for insurance coverage over the 
telephone can easily be resolved by replaying the application call in its entirety. 
Whether a customer selected size 10 can also be proven, as can whether the 
customer/investor authorized the purchase of 100 shares of a particular stock. 
(Eilbacher Col. 9 lines 10-39). 
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Furthermore, Eilbacher teaches types of parameters which can be analyzed by 
the customer experience analyzing unit 208 include the number of key strokes entered 
by the customer during a telephone call, the length of a telephone call, time on hold, 
number of transfers, or length of a queue. That is, if the length of the telephone call, the 
number of key strokes entered during the call or the length of a queue exceeded 
predetermined levels, the customer experience analyzing unit 208 can determine that 
the communication was likely unsatisfactory. In addition, speech detection or word 
spotting can be used to detect certain inflammatory words such as curse words. For 
example, in the case of word spotting, an analysis is performed on recorded audio such 
as a telephone call. The audio is automatically processed, searching for any key words 
on a predefined list which have been identified as cause for concern. If any of the 
words are found, the call is marked as a potentially negative customer experience. This 
word spotting analysis can be done separately, or in addition to the stress analysis. 
Similarly, in connection with an e-mail communication, a text search can be used to look 
for words such as curse words, which might tend to indicate an unsatisfactory customer 
experience (Eilbacher Col. 1 1 lines 25-61 ). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan to 
incorporate verifying that an agent requested a customer's permission to put the 
customer on hold, wherein the pivot spot is the time the agent put the customer on hold, 
the initial region of interest is the whole interaction, and wherein the region of interest is 
defined by a first predetermined number of seconds prior to the pivot spot and a second 
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predetermined number of seconds following the hold as taught by Eilbacher to allow for 
the monitoring of a customer and agent interaction, wherein various retrieval methods 
are implemented based on responses (Eilbacher Col. 8 lines 29-67) and key locations 
of media selected based on the customer responses (i.e. images, keywords, etc.) to 
create and ideal/optimal retrieval of information (Eilbacher Col. 11 lines 25-61). 



6. Claims 10, 14, 15, 17, 18, 26, 30, and 32 rejected under 35 U.S.C. 103(a) as 
being unpatentable over Tsuchikawa et al. US 5748775 A (hereinafter Tsuchikawa) 
in view of Soundararajan US 20030106072 A1 (hereinafter Soundararajan) and 
Eilbacher et al. US 6724887 B1 (hereinafter Eilbacher) further in view of Kung et 
al. US 6917610 B1 (hereinafter Kung). 

Re claim 10, Tsuchikawa in view of Soundararajan and Eilbacher fails to teach 
the apparatus of claim 1 further comprises a computer telephony interface events 
analyzer component to identify and capture at least one common telephony events 
associated with the interaction data 

Kung teaches an activity log may indicate using easy to understand icons as to 
whether the communication is a telephone calls (e.g., record 831), a multimedia video 
and audio calls (e.g., records 832 and 841), a text email (e.g., record 842), and/or an 
instant message email (e.g., record 840), etc. Further, the activity log GUI may be 
include a feature button 802 with a pull down menu that allows the subscriber to select 
between combined media record listing, or one or more of the particular media types 
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such as telephone calls, multimedia video and audio calls, text emails, instant message 
email, etc., to be displayed on the GUI at the same time. In addition, the activity log 
may include records, such as record 833, of incoming communications which have no 
known DN or address associated with them, when the originating party uses, for 
example, a feature such as caller ID blocking (Kung Col. 33 lines 1-25). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan and 
Eilbacherto incorporate invention a computer telephony interface to identify and capture 
common telephony events as taught by Kung to allow for a restricted list of subscribers 
that only have access to phone information (i.e. Caller ID blocking) (Kung Col. 33 lines 
1-25). 

Re claim 14, Tsuchikawa in view of Soundararajan fails to teach the apparatus of 
claim 1 wherein the interaction is one of the following: a telephone call, and e-mail 
message, an audio recording, a video, multimedia data or an interaction media. 

Kung teaches an activity log may indicate using easy to understand icons as to 
whether the communication is a telephone calls (e.g., record 831), a multimedia video 
and audio calls (e.g., records 832 and 841), a text email (e.g., record 842), and/or an 
instant message email (e.g., record 840), etc. Further, the activity log GUI may be 
include a feature button 802 with a pull down menu that allows the subscriber to select 
between combined media record listing, or one or more of the particular media types 
such as telephone calls, multimedia video and audio calls, text emails, instant message 
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email, etc., to be displayed on the GUI at the same time. In addition, the activity log 
may include records, such as record 833, of incoming communications which have no 
known DN or address associated with them, when the originating party uses, for 
example, a feature such as caller ID blocking (Kung Col. 33 lines 1-25). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan and 
Eilbacherto incorporate interaction is one of the following: a telephone call, and e-mail 
message, an audio recording, a video, multimedia data or an interaction media as 
taught by Kung for the purposes of a universal control system fro various multimedia 
equipment/data, wherein all multimedia can interface through a single distributed 
storage media (Kung Col. 33 lines 1-25). 

Re claim 1 5, Tsuchikawa in view of Soundararajan fails to teach the apparatus of 
claim 14 wherein the interaction media is at least one data packet carrying voice or 
other media over internet protocol. 

Kung teaches an IP central station 200 may be configured to provide connectivity 
for the broadband residential gateway 300 to the Internet 180 (e.g., World Wide Web 
(www)), as well as connectivity to other external networks such as public switched 
telephone network 160 and signaling system 7 (SS7) 170 for end-to-end voice, 
multimedia, and data applications, for example voice over IP telephony. IP packets 
traveling through the IP network provide for priority so that, for example, voice packets 
are given priority over data packets to maintain certain VoIP telephony QoS 
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requirements and a leased line concept for packet traffic which may have an even 
higher priority. However, the system is sufficiently flexible so that the priority can be 
dynamically altered according to customer preferences, variable billing rates, traffic 
patterns, and/or congestion (Kung Col. 7 lines 19-33). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan and 
Eilbacher to incorporate data packets carrying voice over IP as taught by Kung to allow 
for the maintenance of telephony operations to assign priority for transmission data, 
wherein various device can be used communicate simultaneously (i.e. phone, PC, etc.) 
(Kung Col. 7 lines 19-33). 

Re claim 17, Tsuchikawa in view of Soundararajan fails to teach the apparatus of 
claim 1 wherein the interaction meta-data (Kung Col. 33 lines 26-39) is an at least one 
computer telephony integrated or CRM event. 

Kung teaches an activity log GUI may have a default set of categories which will 
be displayed to the subscriber, e.g., which determines the type of information such as 
the DN, a system address, an email address 808, a "contacted" party's name 809, a 
company name 810, a client billing number 81 1 , as well as billing information such as 
the date 81 2, time 813, length of the previous communication (duration 815) and any 
associated charges (cost 815) for the previous communication. Further, the GUI may 
include a Category button 804 which enables the user to alter the information displayed 
on the GUI and/or retained in the activity log. Thus, for example the user may alter the 
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activity log to include such as information as a contacted party's street address or 
include alternative DN's or system/email address (Kung Col. 33 lines 1-25). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan and 
Eilbacher to incorporate metadata as taught by Kung to allow for telephony or audio 
communication, wherein information about a conversation or video can be retained to 
allow for indexing of recognizable information such as a pattern or sequence of 
repetitive recorded characteristics (Kung Col. 33 lines 26-39). 

Re claims 18 and 26, Tsuchikawa in view of Soundararajan fails to teach the 
apparatus of claim 1 wherein the interaction meta-data is associated with the at least 
one screen event. 

Kung teaches an activity log GUI may have a default set of categories which will 
be displayed to the subscriber, e.g., which determines the type of information such as 
the DN, a system address, an email address 808, a "contacted" party's name 809, a 
company name 810, a client billing number 81 1 , as well as billing information such as 
the date 812, time 813, length of the previous communication (duration 815) and any 
associated charges (cost 815) for the previous communication. Further, the GUI may 
include a Category button 804 which enables the user to alter the information displayed 
on the GUI and/or retained in the activity log. Thus, for example the user may alter the 
activity log to include such as information as a contacted party's street address or 
include alternative DN's or system/email address (Kung Col. 33 lines 26-39). 



Application/Control Number: 10/706,282 Page 44 

Art Unit: 2626 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan and 
Eilbacher to incorporate interaction meta-data is associated with the at least one screen 
event as taught by Kung to allow for telephony or audio communication with the aid of 
visually seeing past data, wherein information about a conversation or video can be 
retained to allow for indexing of recognizable information such as a pattern or sequence 
of repetitive recorded characteristics (Kung Col. 33 lines 26-39). 

Re claim 30, Tsuchikawa in view of Soundararajan fails to teach the method of 
claim 19 further comprises the steps of: 

identifying an at least one pre-defined computer telephony integrated event in the 
interaction data (Kung Col. 33 lines 1-25); 

identifying an at least one pre-defined screen event in the interaction data (Kung 
Col. 33 lines 26-39) 

Kung teaches an activity log GUI may have a default set of categories which will 
be displayed to the subscriber, e.g., which determines the type of information such as 
the DN, a system address, an email address 808, a "contacted" party's name 809, a 
company name 810, a client billing number 81 1 , as well as billing information such as 
the date 812, time 813, length of the previous communication (duration 815) and any 
associated charges (cost 815) for the previous communication. Further, the GUI may 
include a Category button 804 which enables the user to alter the information displayed 
on the GUI and/or retained in the activity log. Thus, for example the user may alter the 



Application/Control Number: 10/706,282 Page 45 

Art Unit: 2626 

activity log to include such as information as a contacted party's street address or 
include alternative DN's or system/email address. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan and 
Eilbacherto incorporate identifying an at least one pre-defined computer telephony 
integrated event in the interaction data and identifying an at least one pre-defined 
screen event in the interaction data as taught by Kung to allow for telephony or audio 
communication with the aid of visually seeing past data, wherein information about a 
conversation or video can be retained to allow for indexing of recognizable information 
such as a pattern or sequence of repetitive recorded characteristics (Kung Col. 33 lines 
1-25). 

Re claim 32, Tsuchikawa in view of Soundararajan fails to teach the method of 
claim 19 further comprises performing an at least one content analysis step during the 
capturing of the interaction data and the interaction meta-data (Kung Col. 33 lines 26- 
39) 

Kung teaches an activity log GUI may have a default set of categories which will 
be displayed to the subscriber, e.g., which determines the type of information such as 
the DN, a system address, an email address 808, a "contacted" party's name 809, a 
company name 810, a client billing number 81 1 , as well as billing information such as 
the date 812, time 813, length of the previous communication (duration 815) and any 
associated charges (cost 815) for the previous communication. Further, the GUI may 
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include a Category button 804 which enables the user to alter the information displayed 
on the GUI and/or retained in the activity log. Thus, for example the user may alter the 
activity log to include such as information as a contacted party's street address or 
include alternative DN's or system/email address. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan and 
Eilbacher to incorporate performing an at least one content analysis step during the 
capturing of the interaction data and the interaction meta-data as taught by Kung to 
allow for telephony or audio communication with the aid of visually seeing past data, 
wherein information about a conversation or video can be retained to allow for indexing 
of recognizable information such as a pattern or sequence of repetitive recorded 
characteristics (Kung Col. 33 lines 26-39). 

7. Claims 12 and 27-29 rejected under 35 U.S.C. 103(a) as being unpatentable 
over Tsuchikawa et al. US 5748775 A (hereinafter Tsuchikawa) in view of 
Soundararajan US 20030106072 A1 (hereinafter Soundararajan) and Eilbacher et 
al. US 6724887 B1 (hereinafter Eilbacher) and further in view of Chase US 6332143 
B1 (hereinafter Chase). 

Re claims 12 and 27-29, Tsuchikawa in view of Soundararajan and Eilbacher 
fails to teach the apparatus of claim 4, wherein the audio analyzer component further 
comprises the elements of: 



Application/Control Number: 10/706,282 Page 47 

Art Unit: 2626 

a word spotting component to locate and identify pre-defined terms or patterns in 
the speech elements of the interaction data (Chase Col. 17 lines 50-63); 

an emotion analysis component to locate and identify positive or negative 
emotions in the interaction data (Chase Col. 11 lines 45-53); 

a talk analyzer component to identify and locate specific pre-defined speech 
events in the speech elements of the information data (Chase Col. 17 lines 50-63 & Fig. 
5 item 57) 

Chase teaches an emotional descriptor that most closely matches the emotional 
connotation that the connotative judge associates with the word or phrase, considering 
the denotative context and part of speech, wherein a connotative judge understands the 
word or phrase and its denotative context, but does not associate any of the emotional 
descriptors from the supplied list of emotional descriptors with the word or phrase and 
its denotative context. Further, Chase teaches high level information of FIG. 5 includes 
a rating of the passage for emotional content in terms of positive emotion, negative 
emotion and global emotion (i.e., positive not distinct from negative emotion). The high 
level information also includes a rating of the passage for human interest by words, 
sentences and global content, a rating of the passage for power, a rating of the 
passage for activity and a rating of the passage along a scale of abstractness versus 
concreteness (Chase Col. 17 lines 50-63 & Fig. 5 item 57). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan and 
Eilbacherto incorporate a word spotting component to locate and identify pre-defined 
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terms or patterns in the speech elements of the interaction data, an emotion analysis 
component to locate and identify positive or negative emotions in the interaction data, 
and a talk analyzer component to identify and locate specific pre-defined speech events 
in the speech elements of the information data as taught by Chase for the purposes of 
understanding the context of a situation if there is no video available (Chase Col. 17 
lines 50-63 & Fig. 5 item 57). 

8. Claim 40 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Tsuchikawa et al. US 5748775 A (hereinafter Tsuchikawa) in view of 
Soundararajan US 20030106072 A1 (hereinafter Soundararajan) and Eilbacher et 
al. US 6724887 B1 (hereinafter Eilbacher) and further in view of Ronca US 6745221 
B1 (hereinafter Ronca). 

Re claim 40, Tsuchikawa in view of Soundararajan and Eilbacher fails to teach 
the method of claim 19 wherein optimizing the initial region of interest is performed by 
choosing less resource extensive analysis to be performed prior to more resource 
extensive analysis. 

Ronca teaches that a CPU trigger event agent 18 generates CPU trigger events 
when CPU utilization is not available or not at an optimal rate. The resource usage 
trigger event agent 20 generates resource usage trigger events when resources are not 
available to handle applications and it is necessary to re-assign ports to handle the 
applications. The call traffic trigger event agent 22 generates call traffic events when 
the amount of incoming or outgoing traffic is higher or lower than a predetermined 
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threshold level. In the case of incoming or outgoing traffic above the threshold level, the 
call traffic trigger events signify a need for additional port allocation to handle the traffic. 
In the case of incoming or outgoing traffic below the threshold level, the call traffic 
trigger events signify additional port availability. The system clock trigger event agent 
24 generates system clock trigger events at specific times of the day to signal peak and 
non-peak hours as well as business and non-business hours (Ronca Col. 5 lines 43-60). 

Further, Ronca teaches the reassignment of resource allocation by the resource 
allocation manager 10 in response to a call traffic trigger event generated during peak 
hours is illustrated as a representative use case. In this case, high traffic above the 
threshold level is detected by the call traffic trigger event agent 22. The call traffic 
trigger event agent 22 in turn generates a call traffic trigger event 30 and conveys the 
call traffic event to the resource allocation engine 12. When the resource allocation 
engine 12 receives the call traffic trigger event, the resource allocation engine 12 
accesses the resource allocation rules database 16 and checks 32 the resource 
allocation rules therein. Using the rules in Table 1, the resource allocation engine 12 
generates a resource allocation directive 34 for the allocation agents 26 handling the 
ports of group 3. When the resource allocation agents 26 receive the resource 
allocation directive 34, they condition 36 their ports so that they become allocated to 
handle incoming applications (Ronca Col. 6 lines 8-28). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan and 
Eilbacherto incorporate optimizing the initial region of interest is performed by choosing 
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less resource extensive analysis to be performed prior to more resource extensive 
analysis as taught by Ronca to allow for an optimal rate of event handling, wherein 
threshold levels are implemented to handle call traffic in a voice response system to 
allocate an optimal number of agents is available (Ronca Col. 5 lines 43-60). 

9. Claim 43 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Tsuchikawa et al. US 5748775 A (hereinafter Tsuchikawa) in view of 
Soundararajan US 20030106072 A1 (hereinafter Soundararajan) and Eilbacher et 
al. US 6724887 B1 (hereinafter Eilbacher) and further in view of Bernard et al. US 
5918213 A (hereinafter Bernard). 

Re claim 43, Tsuchikawa in view of Soundararajan and Eilbacher fails to teach 
the method of claim 19 wherein the method is used for measuring the effectiveness of a 
promotion offer to a customer requesting the termination of the service, wherein the 
pivot spot is the time of a screen event related to offering a promotion or to an account 
being saved or lost, and wherein the region of interest is defined by a first 
predetermined number of seconds prior to the pivot spot. 

Bernard teaches that promotional items are offered to a customer 182 based on 
his or her calling and purchasing history. For example, in one embodiment, the 
automated product purchasing system review calling and purchasing statistics 
maintained for a shopper 182. Statistics can be maintained by VRU 104, interactive 
transaction database 1 12, or even by reporting database 438. If these statistics indicate 
that the shopper is a particularly good customer of the automated product purchasing 
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system, interface unit 104 may offer a promotional or special item to that shopper 182. 
For example, where shopper 182 is a frequent purchaser, interface unit 104 may inform 
him or her that upon the next purchase, he or she will receive a bonus CD (Bernard Col. 
51 lines 29-41). 

Further, Bernard teaches a caller 182 decides to delete all of the items in his or 
her virtual shopping cart, as indicated by input step 3724, VRU 104 simply deletes these 
items from the order. This deletion step is illustrated by a step 3728. In one 
embodiment this is accomplished by deleting all the order information from the previous 
call and beginning anew with fresh order information for the present call. In an 
alternative embodiment, the deletion is accomplished by simply removing the items' 
catalog ID numbers 1008 from the order information in interactive transaction database 
1 12. A confirmation script can be played by VRU 104 announcing that the order has 
been canceled. As with the other options, at this time the caller is returned to the 
shopping mode where he or she can sample additional selections or terminate the 
phone call. Although not illustrated, in one embodiment, caller 182 is given the option of 
hearing a listing of the items in his or her virtual shopping cart before deciding whether 
to keep or cancel the order entirely. Finally, caller 182 may decide to individually review 
the items in the virtual shopping cart and determine whether each individual item is to 
be kept. If this is the case, caller 182 elects to review the items on hold as illustrated by 
input step 3732. In response, in a step 3736, VRU 104 reviews the order with caller 
182. In one embodiment, this is accomplished by a process similar to that illustrated in 
FIG. 36 where each item is reviewed one at a time, and caller 182 selects whether to 
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accept or delete each item. Once the review order process is complete, caller 182 is 
forwarded to the shopping mode where he or she can sample additional selections, 
immediately purchase the selections remaining in his or her virtual shopping cart, or 
terminate the call (Bernard Col. 50 lines 1-32). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Tsuchikawa in view of Soundararajan and 
Eilbacher to incorporate measuring the effectiveness of a promotion offer to a customer 
requesting the termination of the service, wherein the pivot spot is the time of a screen 
event related to offering a promotion or to an account being saved or lost, and wherein 
the region of interest is defined by a first predetermined number of seconds prior to the 
pivot spot as taught by Bernard to allow for the automated and statistical determination 
of whether a promotional/coupon should be applied to a caller based on the callers 
history (Bernard Col. 51 lines 29-41 ), wherein an order can be cancelled based on a 
caller history (Bernard Col. 50 lines 1-32). 

Conclusion 

10. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. US 6798786 B1 , US 4351 007 A, US 20040001 437 A1 . 

1 1 . Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See M PEP 
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§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michael C. Colucci whose telephone number is (571)- 
270-1847. The examiner can normally be reached on 9:30 am - 6:00 pm, Monday- 
Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on (571)-272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Michael C Colucci/ 
Examiner, Art Unit 2626 
Patent Examiner 
AU 2626 
(571)-270-1847 
Michael.Co iucci@uspto.gov 
/Richemond Dorvil/ 

Supervisory Patent Examiner, Art Unit 2626 



